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REDUCTION  OF  ALL-POLE  PARAMETER  ESTIMATOR  BIAS 
BY  SUCCESSIVE  AUTOCORRELATION 

Darcy  McGinn  and  Don  H.  Johnson 


Abstract 


Conventional,  all-pole  parameter  estimators  applied  to  noise  corrupted  all¬ 


pole  sequences  result  In  biased  estimates.  This  paper  describes  a  procedure  by  which 
reduction  of  this  bias  is  accomplished  by  applying  pole-preserving,  signal  to  noise  ratio 
improving  functions  to  the  sequence.  Correlation  like  pole-preserving  functions  are 
investigated  and  pole  dependent  signal  to  noise  ratio  improvement  is  described.  An 
all-pole  parameter  estimator  using  successive  application  of  a  pole-preserving  function 
(successive  autocorrelation)  is  given.  Comparison  is  made  with  the  least  squares  com¬ 
bination  of  the  higher  order  Yule-Walker  equations,  an  approach  to  bias  reduction 
reported  by  Cadzow.  Successive  autocorrelation  is  found  to  result  in  improved  perfor¬ 
mance,  with  estimates  of  higher  Q  poles  being  most  effectively  enhanced.  ^ 


i 


Introduction 

Goldberger  [1]  and  others  have  shown  that  when  the  classic  least  squares  esti¬ 
mator  of  all-pole  parameters  (Yule-Walker  equations)  is  applied  to  noise  corrupted  all¬ 
pole  sequences,  the  estimates  are  biased.  To  avoid  biased  estimators,  Chan  [2]  applied 
the  method  of  instrumental  variables  to  all-pole  spectrum  estimation.  The  use  of 
lagged  data  as  the  instrumental  variables  results  in  the  higher  order  Yule-Walker  equa¬ 
tions  (HOYWE)  investigated  by  Kay  [3]  and  Gingras  [4].  Kay  noted  that  the  large  vari¬ 
ance  of  estimators  based  on  the  HOYWE  was  a  serious  drawback  to  the  approach.  This 
problem  was  addressed  by  Cadzow  [S]  by  forming  a  least  squares  combination  of  the 
overdetermined  system  of  HOYWE.  The  elements  of  the  final  normal  equations  contain 
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fourth  powera  of  the  data,  and  can  be  thought  of  as  correlations  of  the  correlation  esti¬ 
mates.  Insight  into  then  approaches,  and  a  generalization  of  the  algorithms  used  may 
be  attained  by  viewing  the  elements  of  the  HOYWE  as  a  data  sequence,  and  the  auto¬ 
correlation  estimator  as  a  pole-preserving,  signal  to  noise  ratio  improving  function. 


Problem  Definition 

Let  Sf  be  a  pth  order  all-pole  sequence  given  by  the  difference  equation 


Sf  = 

t=i 

The  noise  corrupted  sequence  is  given  by 


xt  =  st  +  et 


t=  0 . tf-1 


where  €f  is  a:  identically  distributed  zero  mean  noise  sequence  with  variance  c2.  The 
all-pole  sequence  (Sj  ]  is  deterministic  and  represents  the  impulse  response  of  an  all¬ 
pole  digital  *;Tum.  This  model  is  often  used  in  impulse  response  analysis,  speech  pro¬ 
cessing  and  tre  analysis  of  transients. 


Pole-Preserving  Functions 

A  function  of  an  all-pole  sequence  is  pole-preserving  if  the  result  of  applying 
the  function  to  an  all-pole  sequence  is  a  sequence  with  the  same  poles  as  the  original. 
For  example,  the  autocorrelation  of  (s^  j  is  a  pole-preserving  function.  This  becomes 
evident  by  expanding  the  Z  -transform  of  the  autocorrelation  for  lags  ^  0.  The  Z- 
transform  S(z  )  of  a  pth  order  all-pole  sequence  is  given  by 

n; 

s<«)  =  - •  (3) 

4-1 


The  Z  -transform  of  the  autocorrelation  of  (s^  ]  is  given  by 

=  c8i:— t  +  c»£-sL- 

{>1  1-OjZ"1  jall-CLjZ 

The  C{  are  complicated  functions  of  the  b<  and  the  Of.  The  terms  in  s'"1  represent  the 
sequence  \RS  (n )  j  for  n'k 0.  The  Z -transform  of  this  right  sided  sequence  is  given  by 

Zlfl,*(n)i  =  C*£  (4) 

Comparing  (4)  with  (3)  one  notes  that  |i?s+  (n)  j  has  the  same  poles  as  |st  but  the 
modes  have  different  relative  amplitudes.  Stated  another  way.  {/?s+(7l)J  has  a  z- 
transform  having  both  poles  and  zeroes;  the  zeroes  are  due  to  the  modified  values  of 
the  modes.  For  lags  of  p  or  greater.  R^(n  )  obeys  the  same  difference  equation  as  the 
original  sequence. 


Finite  Length  Pole-Preserving  Functions 

All  practical  pole-preserving  functions  will  operate  with  finite  data  lengths. 
One  function  often  used  to  form  the  Yule-Walker  equations  is  the  biased  autocorrela¬ 
tion  estimate  given  by 

N-n-l 

R,(n)  =  £  s*s<+n  n=0,l . N-l  (5) 

t  *0 

Application  to  a  noiseless  all-pole  sequence  (1)  results  in 

R,(n)  =  f;  diRa (n-i)-  f)  s0G6n 

<■1  *■!  tmN-n 

\/ 

Hie  function  has  preserved  the  poles,  but  the  second  term  represents  an  equation 

.•V  •  •  •  *  ■ 

error  which  produces  parameter  estimator  errors  even  in  the  noiseless  case. 
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Another  common  choice  of  correlation  estimator  is  the  "covariance"  type. 


M-l 

~  E  s,s<+n  n= 0.1 . M-M  (e) 

<*o 

where  M  is  the  number  of  terms  in  the  averaging  computation.  Application  to  (1) 
results  in 

=  £)MT,(n-i)  +  s0G6n 

4=1 

Equation  (6)  is  pole-preserving,  and  the  sequence  \Kt  (n )  j  obeys  the  same  difference 
equation  as  (s<  J  for  p  &n£N-M .  Using  the  elements  of  [Kt  (n )  J  in  the  Yule-Walker 
equations  would  add  no  error  to  the  parameter  estimates. 

A  number  of  observations  about  (6):  1)  The  number  of  elements  in  is 

less  U-£s  or  equal  to  the  original  sequence.  2)  The  property  of  satisfying  (l)  for  all  n 
relies  -aiv  on  the  constant  summation  limits,  3)The  St  in  (6)  may  be  replaced  by  any 
length  M  sequence  (vf  ]  which  is  not  orthogonal  to  }  without  affecting  the  pole- 
prese-  vng  property.  A  more  general  form  for  covariance  like  pole-preserving  func¬ 
tions  a 

**,*(n)  =  E  vtst+n  n=0,l . N-M  (7) 

1*0 

Signal  to  Noise  Ratio  Improvement 

The  application  of  (6)  to  {Zj )  gives 

.  .  M-l 

**(*)  =  E 

4*0 


Kz(n)  =  Ks(n)+Z  (n)  n=0,l . N-M  (8) 

\Z(n)l  for  n> 0  is  a  zero  mean  correlated  noise  sequence.  For  n>0  the  variance  of 
Z  (n)  approaches  0  as  M The  averaging  of  Et  in  Z (n)  provides  a  mechanism  for 
signal  to  noise  ratio  improvement.  It  has  been  shown  by  McGinn  [6]  that  the  signal  to 
noise  ratio  improvement  obtained  by  applying  (6)  to  (2)  is  given  approximately  by 
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For  a  1st  order  sequence  this  improvement  is  proportional  to  SNRX .  With  M  —N/2,, 
a =.9,  .AT  =500  and  SNRX  =  10,  SNR/—9ASNRX.  SNR /  increases  for  increasing  Q 

N 

of  the  pole,  for  example,  as  |  a  |  -»1,  SNR[-*~SNRX.  For  complex  pole  pairs, 

analysis  has  shown  that  the  improvement  is  dependent  not  only  on  G.  as  in  the  first 
order  case,  but  also  on  the  angle  of  the  pole.  The  improvement  decreases  as  the  pole 
approaches  the  imaginary  axis.  For  higher  order  sequences  then,  the  modes  with 
highest  Q  and  closest  proximity  to  the  real  axis  are  the  most  strongly  enhanced. 


Since  \Kx(n)]  is  an  all-pole  sequence  corrupted  by  noise,  the  same  form  as 
(2),  the  signal  to  noise  ratio  may  be  improved  further  by  successive  application  of  (6) 
(successive  autocorrelation),  limitations  of  this  approach  include:  the  increasingly 
correlated  nature  of  the  noise  term,  the  increasing  suppression  of  some  of  the  modes 
by  the  zeroes  and  the  smaller  number  of  elements  in  the  sequence.  An  example  of  suc¬ 
cessive  autocorrelation  applied  to  a  sinusoid  in  noise  with  SNRX  of  \/ 10  is  shown  in 
figure  1. 


All-Pole  Parameter  Estimation 


The  above  ideas  can  be  applied  to  all-pole  parameter  estimation  in  the  follow¬ 
ing  algorithm:  1)  Apply  (8)  to  the  noise  corrupted  sequence  J  times  to  obtain  signal  to 
noise  ratio  improvement,  2)  Use  the  standard  covariance  method  to  estimate  the 
parameters  from  the  high  signal  to  noise  ratio  sequence.  Stopping  criteria  for  step  1 
are  based  on  the  limitations  mentioned  above  and  are  discussed  in  [8]. 

The  successive  autocorrelation  concept  was  tested  by  applying  the  above 
algorithm  to  a  noise  corrupted  all-pole  sequence  with  1:0,  1  and  2  with  hi—N/2*. 
The  least  squares  combination  of  the  HOYWE  was  applied  to  the  original  noise  cor¬ 
rupted  sequences  for  comparative  purposes.  The  sequences  were  generated  from  a 
sixth  order  all-pole  model  with  zero  mean  independent  Gaussian  noise  added.  Twenty 
five  realizations  of  length  N=300  and  SNRX~5  were  processed  by  both  algorithms. 
The  results  were  reduced  to  percentage  rms  error  in  frequency  and  radius  for  each 
pole  pair  as  shown  in  table  1.  The  pole  plots  are  shown  in  figures  2  and  3. 


Table  1:  Variance  of  Pole  Estimates 

Processing 

RMS  Error  % 

Method* 

freql 

radl 

freq2 

rad2 

AO 

162.15 

14.49 

17.04 

10.60 

A1 

.48 

.11 

.72 

1.35 

A2 

.57 

.10 

.28 

.57 

B 

.98 

.33 

.70 

1.30 

(8)  N  times, 

B  =  Least  Squares  comb 

ination  o 

AO  is  the  standard  covariance  estimator.  A1  is  similar  to  6  in  that  the  normal  equa¬ 
tions  contain  fourth  moments  of  the  data.  The  normal  equations  in  A2  contain  eighth 
moments  of  the  data  sequence. 


The  pole-location  bias  in  AO  is  evident  in  figure  3a.  This  bias  is  significantly 
reduced  in  the  rest  of  the  methods.  None  of  the  methods  adequately  estimated  the 
lowest  Q  pole.  The  performance  of  A1  and  B  was  comparable,  as  expected.  From  table 
I  it  is  clear  that  procedure  A2  reduced  the  variance  in  the  second  pole-pair  estimate 
and  outperformed  the  HOYWE  approach.  As  a  rule,  the  higher  Q  systems  benefit  more 
from  repeated  correlations.  The  estimation  of  sinusoids  in  noise  is,  therefore,  ideally 
suited  to  this  technique. 


Conclusions 

Successive  application  of  pole-preserving  functions  to  noise  corrupted  all-pole 
sequences  can  increase  the  signal  to  noise  ratio  dramatically.  Conventional  all-pole 
parameter  estimators  applied  to  these  autocorrelated  sequences  show  a  significant 
reduction  in  noise  induced  bias.  Successive  autocorrelation  selectively  improves  the 
parameter  estimates  for  poles  with  highest  Q  and  closest  proximity  to  the  real  axis. 
Sequences  with  these  characteristics  (such  as  sinusoids)  are  most  effectively  enhanced 
by  the  successive  autocorrelation  method.  Since  the  least  squares  combination  of  the 
HOYWE  can  be  interpreted  as  a  successive  autocorrelation  procedure,  this  same  data 
dependent  performance  applies.  Estimates  better  than  the  least  squares  combination 
of  the  HOYWE  can  result  by  allowing  for  two  or  more  correlations,  especially  for  low  sig¬ 
nal  to  noise  ratio. 
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